In order to solve the problems of low accuracy and poor generalization of traffic sign recognition caused by factors such as fog, light, occlusion and large inclination, a lightweight traffic sign recognition method based on neural network was proposed. First, in order to improve image quality, the methods of image normalization, affine transformation and Contrast Limited Adaptive Histogram Equalization (CLAHE) were used for image preprocessing. Second, based on Convolutional Neural Network (CNN), spatial pyramid structure and Batch Normalization (BN) were fused to construct an improved CNN with Spatial Pyramid Pooling (SPP) and BN (SPPN-CNN), and Softmax classifier was used to perform the traffic sign recognition. Finally, the German Traffic Sign Recognition Benchmark (GTSRB) was used to compare the training effects of different image preprocessing methods, model parameters and model structures, and to verify and test the proposed model. Experimental results show that for SPPN-CNN model, the recognition accuracy reaches 98.04% and the loss is less than 0.1, and the recognition rate is greater than 3 000 frame/s under the condition of GPU with low configuration,verifying that the SPPN-CNN model has high accuracy, strong generalization and good real-time performance.